An Oscillator Neural Network Retrieving Sparsely Coded Phase Patterns

Little is known theoretically about the associative memory capabilities of neural networks in which information is encoded not only in the mean firing rate but also in the timing of firings. Particularly, in the case that the fraction of active neurons involved in memorizing patterns becomes small, it is biologically important to consider the timings of firings and to study how such consideration influences storage capacities and quality of recalled patterns. For this purpose, we propose a simple extended model of oscillator neural networks to allow for expression of non-firing state. Analyzing both equilibrium states and dynamical properties in recalling processes, we find that the system possesses good associative memory. 84.35.+i,07.05.Mh,87.10.+e Typeset using REVTEX 1 Since several recent experiments suggest that the temporal coherence of neuronal activity, such as the synchronization of pulses, plays a significant role in real neuronal systems [1], a great number of authors have proposed and studied many theoretical models of neural networks in attempt to understand the essential dynamics of the these systems [2–4]. Among these models, the neural networks of phase oscillators provide a useful framework for modeling and analyzing such temporal behavior in neuronal systems. The main reason for the usefulness of these models is their mathematical tractability, which has allowed us to obtain many important analytic results. Another reason is that, using a certain mathematical technique, the complex dynamics of coupled oscillatory neuronal systems under suitable conditions can be reduced to the dynamics of phase oscillator [5,6]. Since the relation between real systems and these models is theoretically clear, it is expected that their analysis will shed light on the role of oscillatory behavior in real neuronal systems. In particular, the properties of oscillator neural networks with regard to associative memory have been studied recently by many authors [7–11]. With regard to the above mentioned framework, it should be noted that all neurons are assumed to exhibit periodic firing states at all times. However, such an assumption is not realistic from a biological viewpoint. This is because, in real systems, whether or not a neuron is firing usually depends on the situation, in particular the pattern it is presently recalling. In fact, it is well known that only a small fraction of neurons are active at a given time in the central nervous system. Situations in which the level of activity is very low are often termed sparse coding [12]. In the case of standard binary neurons, it is found that the storage capacity for such sparse coding diverges as −1/a ln a, where a is the fraction of active neurons [13–16]. This is the optimal asymptotic form [17]. It is biologically plausible that this theoretical optimal bound is achieved even for a simple Hebbian learning rule. In light of the above considerations, in order to construct more realistic models, it is natural to consider the encoding of information in both the mean firing rate and the relative timing of neuronal spikes. However, little is theoretically known about the properties of associative memory for sparse coded patterns in such systems. For example, one of the interesting questions is whether the storage capacity in such systems, as in the standard binary model, diverges as the activity level a decreases. Another essential point is the quality of the recalled pattern, particularly concerning the timing of the spikes. To clarify these points, we first need to extend the phase oscillator model to allow for expression of the non-firing state [18]. In this paper, we propose and examine a simple extended model as a first step toward the theoretical study of neuronal systems in which the timing of spikes can carry information. Let us start with a brief review of the theoretical basis of the phase oscillator model. It is well known that in coupled oscillatory neuronal systems, under suitable conditions, the original dynamics can be reduced theoretically to a simpler phase dynamics. The state of the ith neuronal oscillatory system can be then characterized by a single phase variable φi representing the timing of the neuronal firings. The typical dynamics of oscillator neural networks are described by equations of the form [5,6,19]. dφi dt = ωi + N